Exploratoring environmental data
RJ
ID
path_to_ncdf## [1] "./DataBaseOllieNew/ID103/Trip_W38/netCDF/"Pressure
Let’s make a graph
### Now without max and min
Some context for each track’s pressure data
Source: Ollie’s notebooks and email exchanges between Jelle, Ollie and me
The data are not calibrated yet.
Since there should be more or less 5-min segments of data every hour, the raw data at 10 Hz are first divided into 5 minute segments (or 15, but it will only contain 5)
Data cleaning is done on these segments. So, each segments:
- Assuming that barometric pressure distributes normally –Jelle said it’s a safe assumption –, outliers are filtered out by first obtaining z-scores (gives you an idea of how far from the mean a data point is. But more technically it’s a measure of how many standard deviations below or above the population mean a raw score is) and keeping the observations whose absolute z-scores are below 3.
- Then, if consecutive observations show a difference greater than 2hPa (meaning a ~20m difference), the following value is replaced by the preceding value.
Resampling is done at 1 Hz. It is basically done by fitting sinusoids to the data (Fast Fourier Transform) and then taking value points. We’re using the obspy.resample function in python which is a wrapper of scipy.signal.resample
Everything is saved into a netCDF file
Time series
## [1] "number of observations is 52840"## [1] "summary diff time Pressure sensor -3.90526986122131"
## [2] "summary diff time Pressure sensor 0.0950310230255127"
## [3] "summary diff time Pressure sensor 0.103679895401001"
## [4] "summary diff time Pressure sensor 5.86721945856235"
## [5] "summary diff time Pressure sensor 0.103681087493896"
## [6] "summary diff time Pressure sensor 3575.79359984398"## [1] "Min, 1st Qu, Median, Mean, 3rdQu, Max"## group median_BaromP mean_BaromP min_BaromP max_BaromP first_Qu_BaromP
## 1 1 1004.6653 1004.6440 1004.2122 1005.3772 1004.5328
## 2 2 1002.8705 1002.8575 1002.1189 1003.7524 1002.7023
## 3 3 1002.4299 1002.4150 1001.7454 1002.9746 1002.2773
## 4 4 1003.1306 1003.1132 1002.5293 1003.4780 1003.0043
## 5 5 1003.4261 1003.4316 1002.9446 1003.9749 1003.2789
## 6 6 1004.2625 1004.2951 1003.8157 1004.7732 1004.1774
## 7 7 1004.2416 1004.2265 1003.7842 1004.6318 1004.0733
## 8 8 1003.7397 1003.7306 1003.1479 1004.0920 1003.6600
## 9 9 1004.3677 1004.3622 1004.0176 1004.7124 1004.2557
## 10 10 1003.9596 1003.9656 1003.7170 1004.3254 1003.8876
## 11 11 1002.9419 1002.9202 1002.3721 1003.3923 1002.7982
## 12 12 1002.7860 1002.7685 1002.3652 1003.1025 1002.6684
## 13 13 1002.3820 1002.3699 1001.7561 1002.7139 1002.2866
## 14 14 1001.7819 1001.7590 1001.1667 1002.2256 1001.6544
## 15 15 998.9103 998.8398 997.9465 999.6138 998.6870
## 16 16 998.1918 998.1742 997.4099 998.7659 998.0139
## 17 17 997.1760 997.1278 996.1445 997.8984 996.9027
## 18 18 995.4299 995.3964 994.6008 995.9795 995.2260
## 19 19 993.3605 993.3499 992.3938 994.0549 993.1247
## 20 20 994.6257 994.6029 993.8801 995.2537 994.4726
## 21 21 995.0159 994.9881 994.2246 995.5679 994.7762
## 22 22 994.5916 994.6104 993.8076 995.4504 994.4024
## 23 23 997.1234 997.1381 996.7314 997.5322 997.0500
## 24 24 998.9180 998.8961 998.3787 999.2344 998.8113
## 25 25 997.4033 997.3355 995.9983 998.3494 997.0769
## 26 26 999.4017 999.3457 997.9536 1000.2871 999.0873
## 27 27 1001.8823 1001.8612 1000.5496 1003.3369 1001.5768
## 28 28 1002.2783 1002.2655 1001.0762 1003.2444 1002.0182
## 29 29 1002.6190 1002.6052 1001.3147 1003.7219 1002.2726
## 30 30 1007.7781 1007.7753 1007.2732 1008.2400 1007.6375
## 31 31 1008.3719 1008.4174 1007.7971 1009.1785 1008.0426
## 32 32 1009.2290 1009.2204 1008.7954 1009.9858 1009.0562
## 33 33 1009.3678 1009.3626 1008.9194 1009.9106 1009.1977
## 34 34 1011.8485 1011.8344 1011.2307 1012.2993 1011.6919
## 35 35 1011.4658 1011.4486 1010.2695 1011.9580 1011.2794
## 36 36 1011.9304 1011.9261 1011.4639 1012.3103 1011.8364
## 37 37 1012.9419 1012.9805 1012.4658 1013.4380 1012.8140
## 38 38 1012.5321 1012.5391 1012.0256 1013.2788 1012.3542
## 39 39 1011.7008 1011.6844 1010.9854 1012.4717 1011.5140
## 40 40 1012.7853 1012.7828 1012.1404 1013.2415 1012.6357
## 41 41 1013.3151 1013.3212 1012.5684 1014.0115 1013.1492
## 42 42 1013.8501 1013.8362 1013.0852 1014.3425 1013.7438
## 43 43 1014.0856 1014.0607 1013.3084 1014.6895 1013.8542
## 44 44 1014.7897 1014.6845 1013.4592 1015.7068 1014.3518
## 45 45 1013.6835 1013.6907 1012.4895 1014.7820 1013.4565
## 46 46 1012.4397 1012.3952 1011.0823 1013.5925 1012.0532
## 47 47 1011.1542 1011.6022 1010.0823 1013.4072 1010.7939
## 48 48 1010.5264 1010.5243 1009.3691 1011.3926 1010.2457
## 49 49 1011.7343 1011.3505 1008.9204 1012.9363 1010.6258
## 50 50 1011.1517 1011.1519 1010.8206 1011.9014 1011.0673
## 51 51 1008.3342 1008.3391 1007.4824 1009.0598 1008.1262
## 52 52 1008.7609 1008.7738 1007.6838 1009.7104 1008.5352
## 53 53 1009.0854 1009.0874 1008.2161 1009.8049 1008.9376
## 54 54 1009.2548 1009.2590 1008.5054 1010.1555 1009.0369
## 55 55 1011.4419 1011.4457 1011.2229 1011.6702 1011.3687
## 56 56 1011.4747 1011.4764 1011.1755 1011.6980 1011.3911
## 57 57 1011.7526 1011.7636 1011.5620 1012.0408 1011.6915
## 58 58 1006.9847 1006.9936 1006.3569 1007.8494 1006.8605
## 59 59 1009.6101 1009.5948 1009.1448 1009.9458 1009.4978
## 60 60 1008.4142 1008.4048 1007.8416 1008.7490 1008.3085
## 61 61 1007.9767 1007.9703 1007.5239 1008.4062 1007.8861
## 62 62 1007.5713 1007.5502 1007.0750 1007.9416 1007.4241
## 63 63 1004.4175 1004.4087 1003.5076 1005.2822 1004.2389
## 64 64 1002.6390 1002.6521 1001.8411 1003.5325 1002.4518
## 65 65 1000.5936 1000.5947 999.9331 1001.3618 1000.3617
## 66 66 1000.7910 1000.7996 1000.1162 1001.5847 1000.7002
## 67 67 1000.6539 1000.8262 999.4133 1002.7859 1000.2472
## 68 68 1001.5499 1001.5547 1000.6243 1002.2144 1001.4041
## 69 69 1002.7961 1002.7633 1001.9321 1003.4392 1002.6199
## 70 70 1003.4053 1003.4384 1002.5566 1004.3945 1003.1971
## 71 71 1003.9114 1003.8865 1003.0679 1004.6426 1003.6039
## 72 72 1003.8633 1003.8667 1003.3010 1004.3906 1003.7192
## 73 73 1004.6531 1004.6062 1003.4670 1005.7766 1004.3568
## 74 74 1004.1903 1004.1938 1003.1802 1005.4719 1003.9152
## 75 75 1004.1471 1004.1104 1002.8108 1005.1462 1003.8542
## 76 76 1005.4268 1005.4445 1005.0120 1006.0896 1005.2820
## 77 77 1004.2435 1004.2773 1003.1672 1005.1206 1004.0133
## 78 78 1006.9924 1006.9947 1006.6199 1007.3301 1006.8511
## 79 79 1007.5394 1007.5438 1007.1772 1007.9468 1007.4238
## 80 80 1008.2933 1008.2945 1007.8618 1008.7136 1008.1820
## 81 81 1008.8339 1008.8643 1008.4202 1009.4573 1008.6703
## 82 82 1009.4902 1009.4666 1009.0164 1009.8796 1009.3362
## 83 83 1009.9581 1009.9592 1009.6191 1010.3491 1009.8644
## 84 84 1010.4524 1010.4235 1009.8118 1011.2412 1010.2355
## 85 85 1010.5406 1010.5107 1010.0559 1010.8503 1010.3958
## 86 86 1010.4788 1010.4633 1009.9944 1010.9248 1010.3486
## 87 87 1008.0751 1008.0451 1007.2651 1008.6812 1007.8428
## third_Qu_BaromP
## 1 1004.7482
## 2 1003.0333
## 3 1002.5707
## 4 1003.2342
## 5 1003.5823
## 6 1004.4196
## 7 1004.3576
## 8 1003.8279
## 9 1004.4574
## 10 1004.0320
## 11 1003.0594
## 12 1002.8802
## 13 1002.4667
## 14 1001.9010
## 15 999.0623
## 16 998.3361
## 17 997.3863
## 18 995.5835
## 19 993.6074
## 20 994.7507
## 21 995.2381
## 22 994.8642
## 23 997.2218
## 24 998.9816
## 25 997.6239
## 26 999.6238
## 27 1002.2001
## 28 1002.5410
## 29 1002.9688
## 30 1007.9195
## 31 1008.7542
## 32 1009.3698
## 33 1009.5032
## 34 1011.9606
## 35 1011.6060
## 36 1012.0357
## 37 1013.1712
## 38 1012.6917
## 39 1011.8504
## 40 1012.9582
## 41 1013.4960
## 42 1013.9590
## 43 1014.3070
## 44 1015.0262
## 45 1013.9606
## 46 1012.7321
## 47 1012.9458
## 48 1010.8339
## 49 1012.0367
## 50 1011.2206
## 51 1008.5444
## 52 1008.9871
## 53 1009.2637
## 54 1009.4696
## 55 1011.5181
## 56 1011.5775
## 57 1011.8287
## 58 1007.1331
## 59 1009.6957
## 60 1008.5134
## 61 1008.0586
## 62 1007.6687
## 63 1004.6107
## 64 1002.8386
## 65 1000.8241
## 66 1000.8992
## 67 1001.0088
## 68 1001.7152
## 69 1002.9146
## 70 1003.7113
## 71 1004.1539
## 72 1004.0121
## 73 1004.8704
## 74 1004.4413
## 75 1004.3860
## 76 1005.5816
## 77 1004.5398
## 78 1007.1339
## 79 1007.6723
## 80 1008.4004
## 81 1009.0342
## 82 1009.5884
## 83 1010.0413
## 84 1010.6050
## 85 1010.6316
## 86 1010.6177
## 87 1008.2537## median_BaromP mean_BaromP min_BaromP max_BaromP first_Qu_BaromP
## 1 1005.435 1005.858 992.3938 1015.707 1002.604
## third_Qu_BaromP
## 1 1010.304{r PressureTS2 , echo=FALSE} ## I was trying to do the plots here... but I have been stuck # ydex=which(Pressure_df3$group<19) # pmedian_BaromP <- ggplot() + geom_point(data=Pressure_df3[ydex,], aes(x=Time3, y= Pressure_hPa)) # geom_hline(aes(yintercept=median_BaromP,color='red')) # pmedian_BaromP #
Infrasound
Some context for each track’s infrasound data
Source: Ollie’s notebooks, email exchanges between Jelle, Ollie and me, some reading and youtube videos
The data are calibrated.
Since there should be more or less 5-min segments of data every hour (could be 1), the raw data at 10 Hz are first divided into 5 minute segments (or 15, but it will only contain 5)
Data cleaning is done on these segments. So, each segments:
- Assuming that absolute pressure distributes normally, outliers are filtered out by first obtaining z-scores and keeping the observations whose absolute z-scores are below 3.
- Then, if consecutive observations show a difference greater than 10 (I don’t know why 10), the following value is replaced by the preceding value.
No resampling
For each segment, the power spectral density is estimated using Welch’s method; it basically divides data into overlapping segments, computing a modified periodogram for each segment and averaging the periodograms
The PSD data are transformed to obtain SPL as \(10 \times \log(PSD/(2*10^{-5})*2))\)
The SPL data are saved into a netCDF file
Time series of Sound Pressure Level from Low Infrasound (0.07-1Hz)
## [1] "number of observations is 57"## [1] "summary diff time Pressure sensor 3602.6380739212"
## [2] "summary diff time Pressure sensor 3603.26880818605"
## [3] "summary diff time Pressure sensor 3603.66190612316"
## [4] "summary diff time Pressure sensor 5213.22480032274"
## [5] "summary diff time Pressure sensor 4504.71238946915"
## [6] "summary diff time Pressure sensor 28891.079985857"